52 research outputs found
A new convergence proof for the higher-order power method and generalizations
A proof for the point-wise convergence of the factors in the higher-order
power method for tensors towards a critical point is given. It is obtained by
applying established results from the theory of \L{}ojasiewicz inequalities to
the equivalent, unconstrained alternating least squares algorithm for best
rank-one tensor approximation
Convergence results for projected line-search methods on varieties of low-rank matrices via \L{}ojasiewicz inequality
The aim of this paper is to derive convergence results for projected
line-search methods on the real-algebraic variety of real
matrices of rank at most . Such methods extend Riemannian
optimization methods, which are successfully used on the smooth manifold
of rank- matrices, to its closure by taking steps along
gradient-related directions in the tangent cone, and afterwards projecting back
to . Considering such a method circumvents the
difficulties which arise from the nonclosedness and the unbounded curvature of
. The pointwise convergence is obtained for real-analytic
functions on the basis of a \L{}ojasiewicz inequality for the projection of the
antigradient to the tangent cone. If the derived limit point lies on the smooth
part of , i.e. in , this boils down to more
or less known results, but with the benefit that asymptotic convergence rate
estimates (for specific step-sizes) can be obtained without an a priori
curvature bound, simply from the fact that the limit lies on a smooth manifold.
At the same time, one can give a convincing justification for assuming critical
points to lie in : if is a critical point of on
, then either has rank , or
On convergence of the maximum block improvement method
Abstract. The MBI (maximum block improvement) method is a greedy approach to solving optimization problems where the decision variables can be grouped into a finite number of blocks. Assuming that optimizing over one block of variables while fixing all others is relatively easy, the MBI method updates the block of variables corresponding to the maximally improving block at each iteration, which is arguably a most natural and simple process to tackle block-structured problems with great potentials for engineering applications. In this paper we establish global and local linear convergence results for this method. The global convergence is established under the Lojasiewicz inequality assumption, while the local analysis invokes second-order assumptions. We study in particular the tensor optimization model with spherical constraints. Conditions for linear convergence of the famous power method for computing the maximum eigenvalue of a matrix follow in this framework as a special case. The condition is interpreted in various other forms for the rank-one tensor optimization model under spherical constraints. Numerical experiments are shown to support the convergence property of the MBI method
Alternating least squares as moving subspace correction
In this note we take a new look at the local convergence of alternating
optimization methods for low-rank matrices and tensors. Our abstract
interpretation as sequential optimization on moving subspaces yields insightful
reformulations of some known convergence conditions that focus on the interplay
between the contractivity of classical multiplicative Schwarz methods with
overlapping subspaces and the curvature of low-rank matrix and tensor
manifolds. While the verification of the abstract conditions in concrete
scenarios remains open in most cases, we are able to provide an alternative and
conceptually simple derivation of the asymptotic convergence rate of the
two-sided block power method of numerical algebra for computing the dominant
singular subspaces of a rectangular matrix. This method is equivalent to an
alternating least squares method applied to a distance function. The
theoretical results are illustrated and validated by numerical experiments.Comment: 20 pages, 4 figure
Finding a low-rank basis in a matrix subspace
For a given matrix subspace, how can we find a basis that consists of
low-rank matrices? This is a generalization of the sparse vector problem. It
turns out that when the subspace is spanned by rank-1 matrices, the matrices
can be obtained by the tensor CP decomposition. For the higher rank case, the
situation is not as straightforward. In this work we present an algorithm based
on a greedy process applicable to higher rank problems. Our algorithm first
estimates the minimum rank by applying soft singular value thresholding to a
nuclear norm relaxation, and then computes a matrix with that rank using the
method of alternating projections. We provide local convergence results, and
compare our algorithm with several alternative approaches. Applications include
data compression beyond the classical truncated SVD, computing accurate
eigenvectors of a near-multiple eigenvalue, image separation and graph
Laplacian eigenproblems
On orthogonal tensors and best rank-one approximation ratio
As is well known, the smallest possible ratio between the spectral norm and
the Frobenius norm of an matrix with is and
is (up to scalar scaling) attained only by matrices having pairwise orthonormal
rows. In the present paper, the smallest possible ratio between spectral and
Frobenius norms of tensors of order , also
called the best rank-one approximation ratio in the literature, is
investigated. The exact value is not known for most configurations of . Using a natural definition of orthogonal tensors over the real
field (resp., unitary tensors over the complex field), it is shown that the
obvious lower bound is attained if and only if a
tensor is orthogonal (resp., unitary) up to scaling. Whether or not orthogonal
or unitary tensors exist depends on the dimensions and the
field. A connection between the (non)existence of real orthogonal tensors of
order three and the classical Hurwitz problem on composition algebras can be
established: existence of orthogonal tensors of size
is equivalent to the admissibility of the triple to the Hurwitz
problem. Some implications for higher-order tensors are then given. For
instance, real orthogonal tensors of order
do exist, but only when . In the complex case, the situation is
more drastic: unitary tensors of size with exist only when . Finally, some numerical illustrations
for spectral norm computation are presented
Dynamical low-rank approximation of the Vlasov-Poisson equation with piecewise linear spatial boundary
We consider dynamical low-rank approximation (DLRA) for the numerical
simulation of Vlasov--Poisson equations based on separation of space and
velocity variables, as proposed in several recent works. The standard approach
for the time integration in the DLRA model uses a splitting of the tangent
space projector for the low-rank manifold according to the separated variables.
It can also be modified to allow for rank-adaptivity. A less studied aspect is
the incorporation of boundary conditions in the DLRA model. We propose a
variational formulation of the projector splitting which allows to handle
inflow boundary conditions on spatial domains with piecewise linear boundary.
Numerical experiments demonstrate the principle feasibility of this approach
On the interconnection between the higher-order singular values of real tensors
A higher-order tensor allows several possible matricizations (reshapes into matrices). The simultaneous decay of singular values of such matricizations has crucial implications on the low-rank approximability of the tensor via higher-order singular value decomposition. It is therefore an interesting question which simultaneous properties the singular values of different tensor matricizations actually can have, but it has not received the deserved attention so far. In this paper, preliminary investigations in this direction are conducted. While it is clear that the singular values in different matricizations cannot be prescribed completely independent from each other, numerical experiments suggest that sufficiently small, but otherwise arbitrary perturbations preserve feasibility. An alternating projection heuristic is proposed for constructing tensors with prescribed singular values (assuming their feasibility). Regarding the related problem of characterising sets of tensors having the same singular values in specified matricizations, it is noted that orthogonal equivalence under multilinear matrix multiplication is a sufficient condition for two tensors to have the same singular values in all principal, Tucker-type matricizations, but, in contrast to the matrix case, not necessary. An explicit example of this phenomenon is given
Maximum relative distance between real rank-two and rank-one tensors
It is shown that the relative distance in Frobenius norm of a real symmetric
order- tensor of rank two to its best rank-one approximation is upper
bounded by . This is achieved by determining the
minimal possible ratio between spectral and Frobenius norm for symmetric
tensors of border rank two, which equals .
These bounds are also verified for arbitrary real rank-two tensors by reducing
to the symmetric case.Comment: New resul
- …